Impact of allocation concealment and blinding in trials addressing treatments for COVID-19: A methods study.

OBJECTIVE: Assess the impact of allocation concealment and blinding on the results of COVID-19 trials. DATA SOURCES: World Health Organization (WHO) COVID-19 database (up to February 2022) Methods: We included randomized trials that compared drug therapeutics with placebo or standard care in patients with COVID-19. We performed random-effects meta-regressions comparing the results of trials with and without allocation concealment and blinding of healthcare providers and patients. RESULTS: We identified 488 trials. We found that, compared to trials with allocation concealment, trials without allocation concealment may estimate treatments to be more beneficial for mortality, mechanical ventilation, hospital admission, duration of hospitalization, and duration of mechanical ventilation, but results were imprecise. We did not find compelling evidence that, compared to trials with blinding, trials without blinding produce consistently different results for mortality, mechanical ventilation, and duration of hospitalization. We found that trials without blinding may estimate treatments to be more beneficial for hospitalizations and duration of mechanical ventilation. CONCLUSION: We did not find compelling evidence that COVID-19 trials in which healthcare providers and patients are blinded produce different results from trials without blinding but trials without allocation concealment estimate treatments to be more beneficial compared to trials with allocation concealment. What's new? Additional information: For decades, allocation concealment (the concealment of the randomization sequence from personnel enrolling participants) and blinding (the concealment of the arm to which participants have been randomized from one or more individuals involved in a trial) have been important considerations in the assessment of risk of bias of trials. Previous studies have produced conflicting results with regards to the associations of blinding and allocation concealment and none have investigated the associations of allocation concealment and blinding in the context of COVID-19. IMPLICATIONS: Our study suggests that lack of blinding may not always bias results but that evidence users should remain skeptical of trials without allocation concealment.

Tactics in Support of the Strategy to Include Artistic Research as Evidence in Systematic Reviews

Artistic research studies produce a felt and often physically embodied type of knowledge initiated in an artistic experience and consolidated as an art form. The actual art work as the outcome of these studies, complements other types of knowledge and therefore requires attention from systematic review authors who synthesize evidence from primary studies. Working with artistic research evidence in a systematic review context requires a different approach to searching, appraising, analysing and integrating research findings than what is usually promoted by international review organisations. In this paper we outline how the different steps in a systematic review process can be adapted to include art work as as a multimodal type of research evidence in systematic reviews. We discuss useful tactics of identifying artistic research evidence, judging its value, analysing and synthesizing such evidence, hereby building on iconographic, thematic and/or art and design related analytical frameworks. In addition, we feature a gallery approach to present artistic research evidence to end-users and feature a multimodal type of evidence synthesis in which individual art works are comprehensively integrated in an audio visual production.

A multifaceted graphical display, including treatment ranking, was developed to aid interpretation of network meta-analysis.

OBJECTIVES: Network meta-analysis (NMA) is becoming a popular statistical tool for analyzing a network of evidence comparing more than two interventions. A particular advantage of NMA over pairwise meta-analysis is its ability to simultaneously compare multiple interventions including comparisons not previously trialed together, permitting intervention hierarchies to be created. Our aim was to develop a novel graphical display to aid interpretation of NMA to clinicians and decision-makers that incorporates ranking of interventions. STUDY DESIGN AND SETTING: Current literature was searched, scrutinized, and provided direction for developing the novel graphical display. Ranking results were often found to be misinterpreted when presented alone and, to aid interpretation and effective communication to inform optimal decision-making, need to be displayed alongside other important aspects of the analysis including the evidence networks and relative intervention effect estimates. RESULTS: Two new ranking visualizations were developed-the 'Litmus Rank-O-Gram' and the 'Radial SUCRA' plot-and embedded within a novel multipanel graphical display programmed within the MetaInsight application, with user feedback gained. CONCLUSION: This display was designed to improve the reporting, and facilitate a holistic understanding, of NMA results. We believe uptake of the display would lead to better understanding of complex results and improve future decision-making.

Conceptualizing the reporting of living systematic reviews.

OBJECTIVES: As part of an effort to develop an extension of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) 2020 statement for living systematic reviews (LSRs), we discuss conceptual issues relevant to the reporting of LSRs and highlight a few challenges. METHODS: Discussion of conceptual issues based on a scoping review of the literature and discussions among authors. RESULTS: We first briefly describe aspects of the LSR production process relevant to reporting. The production cycles differ by whether the literature surveillance identifies new evidence and whether newly identified evidence is judged to be consequential. This impacts the timing, content, and format of LSR versions. Second, we discuss four types of information that are specific to the reporting of LSRs: justification for adopting the living mode, LSR specific methods, changes between LSR versions, and LSR updating status. We also discuss the challenge of conveying changes between versions to the reader. Third, we describe two commonly used reporting formats of LSRs: full and partial reports. Although partial reports are easier to produce and publish, they lead to the scattering of information across different versions. Full reports ensure the completeness of reporting. We discuss the implications for the extension of the PRISMA 2020 statement for LSRs. CONCLUSION: We argue that a dynamic publication platform would facilitate complete and timely reporting of LSRs.

Identifying and managing problematic trials: a Research Integrity Assessment (RIA) tool for randomized controlled trials in evidence synthesis.

Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms to identify and manage problematic studies to avoid misleading findings. As evidence synthesis authors of the Cochrane COVID-19 review on ivermectin, we identified many problematic studies in terms of research integrity and regulatory compliance. Through iterative discussion, we developed a Research Integrity Assessment (RIA) tool for randomized controlled trials (RCTs) for the update of this Cochrane review. In this paper, we explain the rationale and application of the RIA tool in this case study. RIA assesses six study criteria: study retraction, prospective trial registration, adequate ethics approval, author group, plausibility of methods (e.g., randomization), and plausibility of study results. RIA was used in the Cochrane review as part of the eligibility check during screening of potentially eligible studies. Problematic studies were excluded and studies with open questions were held in awaiting classification until clarified. RIA decisions were made independently by two authors and reported transparently. Using the RIA tool resulted in the exclusion of >40% of studies in the first update of the review. RIA is a complementary tool prior to assessing 'Risk of Bias' aiming to establish the integrity and authenticity of studies. RIA provides a platform for urgent development of a standard approach to identifying and managing problematic studies. This article is protected by copyright. All rights reserved.

[Evidence syntheses in public health: An overview]. / Evidenzsynthesen in Public Health: ein Überblick.

BACKGROUND: Since the beginning of the COVID-19 pandemic, there has been a high demand for rapid evidence syntheses to answer urgent public health questions. This article provides an overview of different types of reviews for public health questions and a synthesis of existing recommendations for the preparation of reviews. The aim is to support the planning of one's own review and the critical evaluation of published reviews. METHODS: The basis of this summary is an extensive search for guidelines and recommendations for different review types. Furthermore, internal journal clubs were held to determine knowledge needs and to critically discuss the various review types. For results dissemination, fact sheets were developed for the individual review types including the most important information, prerequisites and work steps, as well as a decision tree for identifying the appropriate review type for the respective question. RESULTS: Of the review types identified, Systematic, Rapid, Scoping, Umbrella, and Narrative Reviews were considered in more detail because they are particularly relevant to public health issues. Together with scoping and umbrella reviews, systematic reviews have the highest resource requirements due to the demands for extensive, systematic evidence synthesis and reproducibility. Rapid methods can accelerate the review process, for example by a very narrowly formulated question, a limited literature search, or the execution of certain steps by one instead of two persons. DISCUSSION: Systematic Reviews may be considered as the gold standard, but they were developed primarily for clinical questions relating to interventions. Instead, this article was focused on review types that consider the diversity of questions as well as the predominant use of quantitative methods in the field of public health. The fact sheets developed and the decision tree should enable low-threshold access to reviews while linking the perspectives of research and resource planning. They complement existing guidelines and recommendations. CONCLUSION: To answer the diverse spectrum of public health questions, various types of reviews with various requirements and approaches are available. Given this diversity, a systematic introduction can be helpful for researchers planning or assessing a review.

Definition, harms, and prevention of redundant systematic reviews.

BACKGROUND: Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy. MAIN BODY: There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers. CONCLUSIONS: Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

The Impacts of COVID-19 Restrictions on Physical Activity in Children and Youth: A Systematic Review of Qualitative Evidence.

BACKGROUND: The objectives of this systematic review were to synthesize qualitative evidence on the impacts of COVID-19 restrictions on physical activity (PA) for children and youth, and explore factors perceived to influence those impacts. METHODS: Five databases (MEDLINE, Embase, SPORTDiscus, ERIC, and CINAHL) were searched initially in June 2021 and updated in December 2021 to locate qualitative articles considering COVID-19 restrictions and PA for children and youth (≤18 y old), in any setting. Eligibility, quality assessments, and data extraction were completed by 2 independent reviewers. Data were synthesized using meta-aggregation with confidence of findings rated using ConQual. RESULTS: After screening 3505 records, 15 studies were included. Curriculum-based PA, organized sport, and active transportation were negatively impacted by COVID-19 restrictions. Negative changes were affected by COVID-19 exposure risks, inadequate instruction, poor access, screen time, and poor weather. Unstructured PA was inconsistently impacted; outdoor unstructured PA increased for some. Positive changes were facilitated by family co-participation, availability of outdoor space, and perceived mental health benefits. CONCLUSION: Qualitative data indicated restrictions had a predominantly negative impact on PA for children and youth, but inconsistent impacts on unstructured PA. The improved contextual understanding offered by our review will be foundational knowledge for health strategies moving forward.

COVID-19 literature surveillance-A framework to manage the literature and support evidence-based decision-making on a rapidly evolving public health topic.

Background: The coronavirus disease 2019 (COVID-19) pandemic has led to a rapid surge of literature on severe acute respiratory syndrome coronavirus 2 and the wider impacts of the pandemic. Research on COVID-19 has been produced at an unprecedented rate, and the ability to stay on top of the most relevant evidence is top priority for clinicians, researchers, public health professionals and policymakers. This article presents a knowledge synthesis methodology developed and used by the Public Health Agency of Canada for managing and maintaining a literature surveillance system to identify, characterize, categorize and disseminate COVID-19 evidence daily. Methods: The Daily Scan of COVID-19 Literature project comprised a systematic process involving four main steps: literature search; screening for relevance; classification and summarization of studies; and disseminating a daily report. Results: As of the end of March 2022 there were approximately 300,000 COVID-19 and pandemic-related citations in the COVID-19 database, of which 50%-60% were primary research. Each day, a report of all new COVID-19 citations, literature highlights and a link to the updated database was generated and sent to a mailing list of over 200 recipients including federal, provincial and local public health agencies and academic institutions. Conclusion: This central repository of COVID-19 literature was maintained in real time to aid in accelerated evidence synthesis activities and support evidence-based decision-making during the pandemic response in Canada. This systematic process can be applied to future rapidly evolving public health topics that require the continuous evaluation and dissemination of evidence.

A Literature Review of the Effects of Air Pollution on COVID-19 Health Outcomes Worldwide: Statistical Challenges and Data Visualization.

Several peer-reviewed papers and reviews have examined the relationship between exposure to air pollution and COVID-19 spread and severity. However, many of the existing reviews on this topic do not extensively present the statistical challenges associated with this field, do not provide comprehensive guidelines for future researchers, and review only the results of a relatively small number of papers. We reviewed 139 papers, 127 of which reported a statistically significant positive association between air pollution and adverse COVID-19 health outcomes. Here, we summarize the evidence, describe the statistical challenges, and make recommendations for future research. To summarize the 139 papers with data from geographical locations around the world, we also present anopen-source data visualization tool that summarizes these studies and allows the research community to contribute evidence as new research papers are published.

The Global Evidence Commission's Report provided a wake-up call for the evidence community.

The Global Evidence Commission report was published in early January 2022. As potential users of the report from France, Lebanon, and South Africa, we have reflected on the value of the report and the next steps for the community. We commend the report for its consideration of a wide range of audiences, and clear recommendations for each. We value its ability to draw lessons from the COVID-19 pandemic, and wider health and non-health fields. The report is an important wake up call for the evidence community and a call to action for us all. Whilst the report is not without its limitations, it is a useful starting point to improve the use of evidence, both in routine times and in future global crises. We all have a role to play in taking forward its recommendations to shape the future of the global evidence movement.

The move towards living systematic reviews and living guidelines in healthcare: consideration of the possibilities and challenges for living qualitative evidence syntheses.

Over the past decade qualitative evidence synthesis (QES), a range of methods for synthesising qualitative research evidence, has become a valued form of evidence for guideline producers who wish to understand more about patient preference and acceptability of treatments. The surge in interest in living systematic reviews and the appearance of living guidelines as a response to the COVID-19 pandemic potentially weaken the value and usability of QES.There are currently no published methods for producing living QES, and if QES are to remain of worth to guideline producers then methods for the rapid, frequent updating of them will need to be developed. We discuss some of the similarities and differences between qualitative and quantitative evidence syntheses and highlight areas where development is needed if reviewers are to progress with living approaches to QES.

Fast Methods for Drug Approval: Research Perspectives for Pandemic Preparedness.

Public heath emergencies such as the outbreak of novel infectious diseases represent a major challenge for drug regulatory bodies, practitioners, and scientific communities. In such critical situations drug regulators and public health practitioners base their decisions on evidence generated and synthesised by scientists. The urgency and novelty of the situation create high levels of uncertainty concerning the safety and effectiveness of drugs. One key tool to mitigate such emergencies is pandemic preparedness. There seems to be, however, a lack of scholarly work on methodology for assessments of new or existing drugs during a pandemic. Issues related to risk attitudes, evidence production and evidence synthesis for drug approval require closer attention. This manuscript, therefore, engages in a conceptual analysis of relevant issues of drug assessment during a pandemic. To this end, we rely in our analysis on recent discussions in the philosophy of science and the philosophy of medicine. Important unanswered foundational questions are identified and possible ways to answer them are considered. Similar problems often have similar solutions, hence studying similar situations can provide important clues. We consider drug assessments of orphan drugs and drug assessments during endemics as similar to drug assessment during a pandemic. Furthermore, other scientific fields which cannot carry out controlled experiments may guide the methodology to draw defeasible causal inferences from imperfect data. Future contributions on methodologies for addressing the issues raised here will indeed have great potential to improve pandemic preparedness.

metaCOVID: A web-application for living meta-analyses of COVID-19 trials.

Outputs from living evidence syntheses projects have been used widely during the pandemic by guideline developers to form evidence-based recommendations. However, the needs of different stakeholders cannot be accommodated by solely providing pre-defined non amendable numerical summaries. Stakeholders also need to understand the data and perform their own exploratory analyses. This requires resources, time, statistical expertise, software knowledge as well as relevant clinical expertise to avoid spurious conclusions. To assist them, we created the metaCOVID application which, based on automation processes, facilitates the fast exploration of the data and the conduct of sub-analyses tailored to end-users needs. metaCOVID has been created in R and is freely available as an R-Shiny application. Based on the COVID-NMA platform (https://covid-nma.com/) the application conducts living meta-analyses of randomized controlled trials related to COVID-19 treatments and vaccines for several outcomes. Several options are available for subgroup and sensitivity analyses. The results are presented in downloadable forest plots. We illustrate metaCOVID through three examples involving well-known treatments and vaccines for COVID-19. The application is freely available from https://covid-nma.com/metacovid/.

A rapid priority setting exercise combining existing, emergent evidence with stakeholder knowledge identified broad topic uncertainties.

OBJECTIVE: The project aimed to rapidly identify priority topic uncertainties as a first step to identify future systematic review questions of pertinence to key international faecal incontinence (FI) stakeholders (patients, carers, healthcare professionals, policy makers and voluntary, community or social enterprise representatives). The paper aim is to share our methods, experience and learning with other groups planning to deliver a rapid priority setting exercise. STUDY DESIGN: An evidence gap map incorporated three evidence streams: emerging evidence identified through horizon scanning; existing evidence identified through systematic searches of bibliographic databases; and FI stakeholder insights collected through an international survey. The evidence gap map was presented during an online workshop with stakeholders, where they shared their expertise to expand, refine and rank topic uncertainties using ideation techniques, focus group discussions, consensus techniques and online polling. RESULTS: The multi-step methods used to deliver this priority-setting exercise resulted in identification of broad priority topic uncertainties. The methods appear to have high acceptability and engagement with participants but await full evaluation. CONCLUSIONS: This project successfully followed robust methodology, building upon frameworks from published priority setting and evidence gap mapping projects whilst incorporating strong patient and public involvement components.

From standard systematic reviews to living systematic reviews.

Systematic reviews (SRs) have become a central tool for evidence-based health care over the last 30 years. The number of SRs being published has increased steadily. However, concerns have been raised regarding the duplication of work, methodological flaws and the currency of many systematic reviews, also in the context of the COVID-19 pandemic. Living systematic reviews (LSRs) offer a new approach to updating systematic reviews, particularly in high-priority research fields that face the challenge of dynamically evolving and sometimes uncertain evidence. Continual updates serve to ensure that LSRs remain current and methodologically rigorous. As a new element of the evidence ecosystem, LSRs can inform living guidelines and recommendations, user-adapted formats, decisions at the patient and system level as well as gaps in primary research.

Unit information prior for incorporating real-world evidence into randomized controlled trials.

Randomized controlled trials (RCTs) have been widely recognized as the gold standard to infer the treatment effect in clinical research. Recently, there has been growing interest in enhancing and complementing the result in an RCT by integrating real-world evidence from observational studies. The unit information prior (UIP) is a newly proposed technique that can effectively borrow information from multiple historical datasets. We extend this generic approach to synthesize the non-randomized evidence into a current RCT. Not only does the UIP only require summary statistics published from observational studies for ease of implementation, but it also has clear interpretations and can alleviate the potential bias in the real-world evidence via weighting schemes. Extensive numerical experiments show that the UIP can improve the statistical efficiency in estimating the treatment effect for various types of outcome variables. The practical potential of our UIP approach is further illustrated with a real trial of hydroxychloroquine for treating COVID-19 patients.

From Centuries to Hours: The Journey of Research into Practice

COVID-19 has made the research-policy nexus visible as never before. Public health officials, doctors and scientists are standing alongside political and other leaders all around the world to promote research-informed health behaviors to ‘flatten the curve' of this global pandemic. Proper hand hygiene, physical distancing and use of masks are now either recommended or mandated everyday public health behaviors across the world. Obviously, application of research to public problem solving is not new. However, the pace of the journey from research to policy and practice has shortened at a similar exponential rate as the spread of the virus. This paper provides a historical overview of research-informed policy and practice. In doing so, it aims to build an understanding of how research is able to address COVID-related policy questions in almost real-time;how the demand-side consequences of this global pandemic have advanced research-informed policy and practice;and how policymakers can harness research to solve public policy problems in the future. © 2022 Copyright held by the owner/author(s).

The Network of University Medicine: Transferring New COVID-19 Research Findings into Patient Care

Guidelines are a key tool in the dissemination of new research findings for patient treatment. They are structured to support the best possible evidence-based care. For the management of a new disease such as COVID-19, for which there was initially no established care practice, the rapid development of up-to-date guidelines was essential. In view of intensive worldwide research, a great deal of new knowledge has been generated in a short time. At the same time, a large number of patients required acute care. The wealth of new knowledge thus had to be transferred very quickly into practice at a time when medical staff were already heavily burdened by the high demand for care. Furthermore, the preparation of guidelines is time-consuming and costly. For this reason, the Network University Medicine (NUM), a research association of 36 German university hospitals, has developed approaches in the CEOsys project to significantly accelerate the transfer of new findings from COVID-19 research into patient treatment. The platforms required for this must now be consolidated in order to permanently improve pandemic preparedness in Germany. Copyright © 2022 Georg Thieme Verlag. All rights reserved.

Thirty years of research on physical activity, mental health, and wellbeing: A scientometric analysis of hotspots and trends.

The sheer volume of research publications on physical activity, mental health, and wellbeing is overwhelming. The aim of this study was to perform a broad-ranging scientometric analysis to evaluate key themes and trends over the past decades, informing future lines of research. We searched the Web of Science Core Collection from inception until December 7, 2021, using the appropriate search terms such as "physical activity" or "mental health," with no limitation of language or time. Eligible studies were articles, reviews, editorial material, and proceeding papers. We retrieved 55,353 documents published between 1905 and 2021. The annual scientific production is exponential with a mean annual growth rate of 6.8% since 1989. The 1988-2021 co-cited reference network identified 50 distinct clusters that presented significant modularity and silhouette scores indicating highly credible clusters (Q = 0.848, S = 0.939). This network identified 6 major research trends on physical activity, namely cardiovascular diseases, somatic disorders, cognitive decline/dementia, mental illness, athletes' performance, related health issues, and eating disorders, and the COVID-19 pandemic. A focus on the latest research trends found that greenness/urbanicity (2014), concussion/chronic traumatic encephalopathy (2015), and COVID-19 (2019) were the most active clusters of research. The USA research network was the most central, and the Chinese research network, although important in size, was relatively isolated. Our results strengthen and expand the central role of physical activity in public health, calling for the systematic involvement of physical activity professionals as stakeholders in public health decision-making process.